The role of decision tree and machine learning models for outcome prediction of bupropion exposure: A nationwide analysis of more than 14 000 patients in the United States.

Bupropion is widely used for the treatment of major depressive disorder and for smoking cessation assistance. Unfortunately, there are no practical systems to assist clinicians or poison centres in predicting outcomes based on clinical features. Hence, the purpose of this study was to use a decision tree approach to inform early diagnosis of outcomes secondary to bupropion overdose. This study utilized a dataset from the National Poison Data System, a 6-year retrospective cohort study on toxic exposures and patient outcomes. A machine learning algorithm (decision tree) was applied to the dataset using the sci-kit-learn library in Python. Shapley Additive exPlanations (SHAP) were used as an explainable method. Comparative analysis was performed using random forest (RF), Gradient Boosting classification, eXtreme Gradient Boosting, Light Gradient Boosting (LGM) and voting ensembling. ROC curve and precision-recall curve were used to analyse the performance of each model. LGM and RF demonstrated the highest performance to predict outcome of bupropion exposure. Multiple seizures, conduction disturbance, intentional exposure, and confusion were the most influential factors to predict the outcome of bupropion exposure. Coma and seizure, including single, multiple and status, were the most important factors to predict major outcomes.

Classification of acute poisoning exposures with machine learning models derived from the National Poison Data System.

The primary aim of this pilot study was to develop a machine learning algorithm to predict and distinguish eight poisoning agents based on clinical symptoms. Data were used from the National Poison Data System from 2014 to 2018, for patients 0-89 years old with single-agent exposure to eight drugs or drug classes (acetaminophen, aspirin, benzodiazepines, bupropion, calcium channel blockers, diphenhydramine, lithium and sulfonylureas). Four classifier prediction models were applied to the data: logistic regression, LightGBM, XGBoost, and CatBoost. There were 201 031 cases used to develop and test the algorithms. Among the four models, accuracy ranged 77%-80%, with precision and F1 scores of 76%-80% and recall of 77%-78%. Overall specificity was 92% for all models. Accuracy was highest for identifying sulfonylureas, acetaminophen, benzodiazepines and diphenhydramine poisoning. F1 scores were highest for correctly classifying sulfonylureas, acetaminophen and benzodiazepine poisonings. Recall was highest for sulfonylureas, acetaminophen, and benzodiazepines, and lowest for bupropion. Specificity was >99% for models of sulfonylureas, calcium channel blockers, lithium and aspirin. For single-agent poisoning cases among the eight possible exposures, machine learning models based on clinical signs and symptoms moderately predicted the causal agent. CatBoost and LightGBM classifier models had the highest performance of those tested.

A clinician survey of using speech recognition for clinical documentation in the electronic health record.

OBJECTIVE: To assess the role of speech recognition (SR) technology in clinicians' documentation workflows by examining use of, experience with and opinions about this technology. MATERIALS AND METHODS: We distributed a survey in 2016-2017 to 1731 clinician SR users at two large medical centers in Boston, Massachusetts and Aurora, Colorado. The survey asked about demographic and clinical characteristics, SR use and preferences, perceived accuracy, efficiency, and usability of SR, and overall satisfaction. Associations between outcomes (e.g., satisfaction) and factors (e.g., error prevalence) were measured using ordinal logistic regression. RESULTS: Most respondents (65.3%) had used their SR system for under one year. 75.5% of respondents estimated seeing 10 or fewer errors per dictation, but 19.6% estimated half or more of errors were clinically significant. Although 29.4% of respondents did not include SR among their preferred documentation methods, 78.8% were satisfied with SR, and 77.2% agreed that SR improves efficiency. Satisfaction was associated positively with efficiency and negatively with error prevalence and editing time. Respondents were interested in further training about using SR effectively but expressed concerns regarding software reliability, editing and workflow. DISCUSSION: Compared to other documentation methods (e.g., scribes, templates, typing, traditional dictation), SR has emerged as an effective solution, overcoming limitations inherent in other options and potentially improving efficiency while preserving documentation quality. CONCLUSION: While concerns about SR usability and accuracy persist, clinicians expressed positive opinions about its impact on workflow and efficiency. Faster and better approaches are needed for clinical documentation, and SR is likely to play an important role going forward.

Analysis of Errors in Dictated Clinical Documents Assisted by Speech Recognition Software and Professional Transcriptionists.

IMPORTANCE: Accurate clinical documentation is critical to health care quality and safety. Dictation services supported by speech recognition (SR) technology and professional medical transcriptionists are widely used by US clinicians. However, the quality of SR-assisted documentation has not been thoroughly studied. OBJECTIVE: To identify and analyze errors at each stage of the SR-assisted dictation process. DESIGN SETTING AND PARTICIPANTS: This cross-sectional study collected a stratified random sample of 217 notes (83 office notes, 75 discharge summaries, and 59 operative notes) dictated by 144 physicians between January 1 and December 31, 2016, at 2 health care organizations using Dragon Medical 360 | eScription (Nuance). Errors were annotated in the SR engine-generated document (SR), the medical transcriptionist-edited document (MT), and the physician's signed note (SN). Each document was compared with a criterion standard created from the original audio recordings and medical record review. MAIN OUTCOMES AND MEASURES: Error rate; mean errors per document; error frequency by general type (eg, deletion), semantic type (eg, medication), and clinical significance; and variations by physician characteristics, note type, and institution. RESULTS: Among the 217 notes, there were 144 unique dictating physicians: 44 female (30.6%) and 10 unknown sex (6.9%). Mean (SD) physician age was 52 (12.5) years (median [range] age, 54 [28-80] years). Among 121 physicians for whom specialty information was available (84.0%), 35 specialties were represented, including 45 surgeons (37.2%), 30 internists (24.8%), and 46 others (38.0%). The error rate in SR notes was 7.4% (ie, 7.4 errors per 100 words). It decreased to 0.4% after transcriptionist review and 0.3% in SNs. Overall, 96.3% of SR notes, 58.1% of MT notes, and 42.4% of SNs contained errors. Deletions were most common (34.7%), then insertions (27.0%). Among errors at the SR, MT, and SN stages, 15.8%, 26.9%, and 25.9%, respectively, involved clinical information, and 5.7%, 8.9%, and 6.4%, respectively, were clinically significant. Discharge summaries had higher mean SR error rates than other types (8.9% vs 6.6%; difference, 2.3%; 95% CI, 1.0%-3.6%; P < .001). Surgeons' SR notes had lower mean error rates than other physicians' (6.0% vs 8.1%; difference, 2.2%; 95% CI, 0.8%-3.5%; P = .002). One institution had a higher mean SR error rate (7.6% vs 6.6%; difference, 1.0%; 95% CI, -0.2% to 2.8%; P = .10) but lower mean MT and SN error rates (0.3% vs 0.7%; difference, -0.3%; 95% CI, -0.63% to -0.04%; P = .03 and 0.2% vs 0.6%; difference, -0.4%; 95% CI, -0.7% to -0.2%; P = .003). CONCLUSIONS AND RELEVANCE: Seven in 100 words in SR-generated documents contain errors; many errors involve clinical information. That most errors are corrected before notes are signed demonstrates the importance of manual review, quality assurance, and auditing.

Incidence of speech recognition errors in the emergency department.

BACKGROUND: Physician use of computerized speech recognition (SR) technology has risen in recent years due to its ease of use and efficiency at the point of care. However, error rates between 10 and 23% have been observed, raising concern about the number of errors being entered into the permanent medical record, their impact on quality of care and medical liability that may arise. Our aim was to determine the incidence and types of SR errors introduced by this technology in the emergency department (ED). SETTING: Level 1 emergency department with 42,000 visits/year in a tertiary academic teaching hospital. METHODS: A random sample of 100 notes dictated by attending emergency physicians (EPs) using SR software was collected from the ED electronic health record between January and June 2012. Two board-certified EPs annotated the notes and conducted error analysis independently. An existing classification schema was adopted to classify errors into eight errors types. Critical errors deemed to potentially impact patient care were identified. RESULTS: There were 128 errors in total or 1.3 errors per note, and 14.8% (n=19) errors were judged to be critical. 71% of notes contained errors, and 15% contained one or more critical errors. Annunciation errors were the highest at 53.9% (n=69), followed by deletions at 18.0% (n=23) and added words at 11.7% (n=15). Nonsense errors, homonyms and spelling errors were present in 10.9% (n=14), 4.7% (n=6), and 0.8% (n=1) of notes, respectively. There were no suffix or dictionary errors. Inter-annotator agreement was 97.8%. CONCLUSIONS: This is the first estimate at classifying speech recognition errors in dictated emergency department notes. Speech recognition errors occur commonly with annunciation errors being the most frequent. Error rates were comparable if not lower than previous studies. 15% of errors were deemed critical, potentially leading to miscommunication that could affect patient care.